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ABSTRACT 

An evaluation was done of New York City Public 
Schools' Dropout Reduction through Employment, Achievement, and 
Motivation Program (Project DREAM) , which served poor 
Spanish- speaking immigrant high school students. The program operated 
at South Bronx High School serving 444 mostly immigrant students of 
limited English proficiency. The program offered English as a second 
Language (ESL) , native language arts (NLA) , and bilingual instruction 
in content areas. A program strength was small class size. Two 
guidance counselors, a case manager, and a community-based 
organization provided services to participants. Evaluation of the 
program was based on demographic data, citywide student test scores, 
and interviews with and surveys of the program director. Project 
DREAM was fully implemented; and it met its objective for NLA, career 
education, attendance, staff development, and parental ii. .olvement . 
It met one of two ESL objectives and two of three content area 
objectives. In addition, parental involvement was found to be low and 
in need of strengthening. Two appendices summarize the data 
collection and analysis methods and list instructional materials. 
(JB) 



* Reproductions supplied by EDRS are the best that can be made 

* from the original document. 



HHB ORE A Report 



Dropout Reduction through Employment 
Achievement and Motivation 

(Project DREAM) 
Grant Number: T003A90063 

1990-91 



FINAL EVALUATION PROFILE 



■•PERMISSION TO REPRODUCE THIS 
MATERIAL HAS BEEN GRANTED BY 



rERJAL HAS I 

ft To. 



TO THE EDUCATIONAL RESOURCES 
INFORMATION CENTER (ERICV* 




EDUCATE I 

C Mll >of Chanel 6W> m « <,p »° ' mp,ov8 
loptoOuctior' ouawy 



BEST COPY AVAILABLE 



Ik 



Dropout Reduction through Employment 
Achievement and Motivation 

(Project DREAM) 
Grant Number: T003A90063 

1990-91 

FINAL EVALUATION PROFILE 



o 

ERIC 



3 



NEW YORK CITY BOARD OF EDUCATION 



H. Carl McCaU 
President 

Irene H. Impellizzeri 
Vice President 

Carol A. Cresser 
Weaina L. Matthews 
Michael J. Petridcs 
Luis 0. Reyes 
Ninfa Segarra 
Members 

Joseph A. Fernandez 
Chancellor 



DIVISION OF STRATEGIC PLANNING/RESEARCH & DEVELOPMENT 

Robin Winner 
Executive Director 



it is the poiky of the New York City Board of Fdocapon not to docrgmani on the beta of race, color, creed, religion. 
Mtjontlonfjn.«fe.h*ndk*ppmtcoodii^ 

»nd employ moit pobciee, «» reqwrtd by lew. laouiriee rtgwdini compttaoce with tnnopriate lew* miy be directed u> 
£J" c * k *A N *!™ D™*** O^ 6 * ^ &p»l Opportunity. HO Liviofsoa Street Brooklyn. New York 11201; or 10 
Director. Office for OP Ri&t*. Department of Edoctoon. 26 Fedaii »taa. Room 33-10. New York. New York 10278. 



7/28/91 



ACKNOWLEDGMENTS 



This report has been prepared by the Bilingual, Multicultural, and Early Childhood Evaluation Un 

of the Office of Research, Evaluation, and Assessment. 

Additional copies of this report are available from: 

Dr. Tomi Deutsch Berney 

Office of Research, Evaluation, and Assessment 

New York City Public Schools 

110 Livingston Street, Room 732 

Brooklyn, NY 11201 

(718) 935-3790 



FOREWORD 



The body of this report is preceded by an Extract which presents an overview of salient points of 
the project: funding cycle; enrollment figures; background of students served; admission criteria; and 
programming features, strengths, and limitations, including the outcome of all objectives. The extract also 
presents the conclusions drawn by the Office of Research, Evaluation, and Assessment (OREA) about the 
program and its recommendations for program improvement. 

The extract is followed by the body of the report, titled Procram Assessmen t. This includes such 
information as staffing, program implementation, and outcome and implementation objectives. 
Instructional objectives are presented first, followed by noninstructional objectives. The report then 
addresses those aspects of programming mandated by Title VII regulations that do not have specifically 
stated objectives. This may be information on attendance and dropout rate, grade retention, 
mainstreaming, referrals out of the program to meet special needs of the students, and withdrawals. A 
case history concludes the report. 

Data for this profile were collected and analyzed using a variety of procedures, which are 
described in A ppendix A following the text. 



Office of Research, Evaluation and Assessment 
New York City Public Schools 

E.S.E.A. Title VII Evaluation Profile 

Dropout Reduction through Employment, 
Achievement, and Motivation 
Project DREAM 

Grant Number: T003A90063 
1990-91 

EXTRACT 

PROJECT DIRECTOR; Sheila Fallick FUNDING CYCLE: Year 2 of 3 

SITES 

School Borough Grade Levels Enrollment* 



(fell) (spring) 

South Bronx High School Bronx 9- 11 414 334 

•The project enrolled 444 students (1 less than in the previous year). 
STUDENT BACKGROUND 

Native Language Number of Students Countries of Origin Number of Students 

Spanish 439 Puerto Rico 121 

Unreported 5 Dominican Republic 175 

Unites States 37 

Ecuador 26 

Honduras 26 

Mexico 24 

Other countries 35 



Students lived in poor economic conditions and many had after-school jobs and child-care responsibilities. 
The previous education of project students from South America and Puerto Rico was fair; the students from 
Central America and the Dominican Republic had a poor educational background. 

Mean Years of Education in Native Country : 6.5; in the United States: 3J 



ADMISSION CRITERIA 

The project admitted students who scored at or below the 40tb percentile on the Language Assessment 
Battery (LAB) and were therefore categorized as being of limited English proficiency (LEP). It also admitted 
students who opted into the program through parental authorization. 

PROGRAMMING 

Features 

Project DREAM targeted one school with a recent influx of immigrants. The program offered English as 
a Second Language (E.S.L.), Native Language Arts (N.L.A), and bilingual instruction in content areas. 
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Strengths and Limitations 

One of the program's strengths was its small classes. lis addition, two guidance counselors, a case 
manager, and ASPIRA-a community-based organization--, provided services to program students. 

Parent involvement was low and in need of strengthening. 

CONCLUSIONS AND RL COMMENDATIONS 

Project DREAM was fully implemented. The program met its objective for N.L.A., career education, 
attendance, staff development, and parental involvement. It met one of two E.S.L. objectives, Of the three 
content area objectives, the project met two and partially met the third The self-image objective could not be 
evaluated for lack of data. 

The conclusions, based on the findings of thus evaluation, lead to the following recommendations: 

• Submit data nect ssary for OREA to evaluate all objectives. 

• Explore reasons why project students were unable to meet the E.S.L. objective for gains in language 
skills. 

• Explore reasons why passing rates for the R.C.T. in global studies and science were low. 
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PROGRAM ASSESSMENT 



STAFFING 



Title VII (Total 6) 



Tjile 

Project Director 
Bil. Resource Teacher 
Case Manager 
Bil. Ed Assistant 
Bil. Ed. Assistant 
Bil. Ed Assistant 



Degree 

MA 

M.A. 

M.S. 

B.A. 



Language Competencies 
Spanish, French 



Spanish 
Spanish 
Spanish 
Spanish 
Spanish 



Other Staff Working With Project Students (Total 28) 



Degree 
M-A. 16 
B.A. 12 



Certification 
E.S.L. 



4 
4 



Language Competencies 
Spanish 18 



Mathematics 
Science 



Social Studies 
Unreported 



4 
15 



IMPLEMENTATION AND OUTCOMES (objectives prefaced by •) 

The project used educational assistants and computers to assist instruction in all academic subjects. 
English as a Second Language (E.S.L.) 



For a list of textbooks see Appendix B. 

• Seventy-five percent of the program students will continue to demonstrate gains in achievement in 
language skills (reading, writing, listening, and speaking). 

Evaluation Instrument: Language Assessment Battery (LAB) * 

Pretest: March 1990; posttest: March 1991. 



OREA used a gap reduction design to evaluate the effect of supplementary instruction on project students' 
performance on the LAB. Since all LEP students in New York City are entitled to such instruction, no valid 
comparison group exists among these students, and OREA used instead the group on which the LAB was 
normed. Test scores are reported in Normal Curve Equivalents (N.CE-s), which are normalized standard scores 
with a mean of SO and a standard deviation of 21.06. It is assumed that the norm group has a zero gain in 
N.C.E.s in the absence of supplementary instruction and that participating students' gains are attributable to 
project services. 



Level 



Periods Weekly 



Beginning 
Intermediate 
Advanced 
Transitional 



15 
15 
10 
5 
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Number of students for whom pre- and posttcst data were reported: 272 
Percentage of students showing gain: 67.3 
Project did not meet objective. 

• Seventy percent of the program students will score at the passing criterion of 65 in E.S.L. 
and E.S.L reading. 

Evaluation Indicator: Final course grades. 

Passing Grades in E.S.l r < Courses 

FaU Spring 

Enrollment Percent Passing Enrollment Percent Passing 

E.S.L. 263 79.8 294 70 .1 

E.S.L. Reading/Writing 311 70.1 231 74.5 

Over 70 percent of students received final grades of at least 65 in E.S.L. and E.S.L. reading 
in both semesters. 

Project met objective. 

Native Language Arts (N.L.A.1 

Estimated percentage of students lacking literacy skills in native language: 10 

Leve| Periods Weekly 

Literacy 10 
Beginning 5 
Intermediate 5 
Advanced 5 
Adv. Placement 5 

For a list of instructional materials used, see Appendix B. 

• Seventy-five percent of the target students will demonstrate significant gains in the Spanish language 
skills of reading and writing. 

Evaluation Instrument: Program-made test. 

Number of students for whom pre* and posttest data were reported: 1 1 

Percentage of students showing gain: 90.9 

This test was only administered to those students at the literacy level 
Project met objective. 



in 



• Seventy percent of the target students will score at or above the passing criterion of 65 in Native 
Language Arts each semester. 

Evaluation Indicator: Final course grades. 

Of the 404 students enrolled in N.L.A. in the fail 78.2 percent (316) passed. Of the 398 students 
enrolled in N.i-A. in the spring, 76.3 percent (304) passed. 

Project met objective. 
Content Area Subjects 

Courses in mathematics, science, and social studies were taught in the students' native language. Program 
students had particular difficulty with social studies due to their lack of reading skills* 

• Students participating in the program will score at or above the passing criterion of 65 in bilingual 
mathematics, science, and social studies classes at a rate that is equal to that of non-program students 
in comparable mainstream classes. 

Evaluation Indicator: Final course grades. 

Passing Grades in Content Area Subjects 



Project Students Mainstream Students 

Number Percentage Percentage 

Mathematics 225 70.1 49.1 

Science 230 76.1 51.8 

Social studies 276 73.9 583 



Project met objective. 

• The percentage of target students passing the Regents examinations in Sequential Mathematics and 
Biology will be equal to that of mainstream students. 

Evaluation Instrument: New York State Regents examination scores. 

Of the 24 program students taking the Sequential Mathematics I Regents examination, 100 percent 
passed. Of the 14 mainstream students taking the exam, four (28,6 percent) passed. No project students took the 
Biology Regents examination this year. 

Project met objective- 
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The percentage of target students passing the Regents Competency Tests (R.C.T.s) in reading, 
writing, mathematics, science, and global history will be equal to that of mainstream students. 



Evaluation Instrument: Regents Competency Test results. 



Passing Grades on Regents Competency Tests 



Project Students 



Mainstream Students 



Number Percent Passing 



Number Eercent Passing 



R.C.T. Mathematics 
R.C.T. Science 



112 
55 
64 

30 



60.7 
41.8 
39.0 
96.7 



101 
112 
44 
42 



54.4 
42.8 
56.8 
83.3 



R.C.T. Global Studies 
R.C.T. U.S. History 



The percentage of project students who passed the R.C.T. in mathematics and U.S. History exceeded that 
of percentage of mainstream students. The percentages of project and mainstream students who passed the 
R.C.T. in science were very similar. However, a greater percentage of mainstream than project students passed 
the R.C.T. in global studies. 

Project partially met objective. 



Attendance 

• Students in the program will maintain a higher rate of attendance than that of mainstream students. 

The attendance rate for project students was 89.2 percent; that for mainstream siudents was 80.8 percent. 
The rate for project siudents was higher. 

Project met objective. 



Dropout 

Eleven project students (2.5 percent) dropped out. The project does not call for an objective for dropout 
rate until 1992. 



Self-Image 

• Seventy percent of the program students will develop a more positive self-image. 
Evaluation Instrument: School Attitude Measure. 

The project submitted to OREA an analysis which had been performed by a third party, and which 
determined that project students showed statistically significant improvement in five dimensions of self-image. 
The analysis did not indicate, however, what percentage of students demonstrated improvement. 

OREA could not evaluate objective. 
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Career Education 



• Students will set attainable goals for their future through career education infujion in E.S.L, classes 
and guidance sessions with the case manager. 

Individual and group sessions on career goals were offered by the case manager. Teachers in the E,S.L. 
department also offered career education counseling in their classes. It was a part of the curriculum to discuss 
career goals in the class. 

Project met objective. 

Staff Development 

• All staff members will be fully trained in educational aspects related to E.S.L., bilingual education, 
native language arts, and bilingual content areas through college courses, workshops, seminars, and 
conferences. 

All project staff attended monthly faculty and departmental meetings and a full day staff development 
workshop on "Instructional Strategies Used with LEP Students/ Various staff also attended E-S.L and United 
Federation of Teachers {UFT>sponsored conferences, the Teachers of English to Speakers of Other Languages 
(TESOL) convention, and the bilingual education staff development program. Eight program teachers enrolled in 
college courses during the school year. 

Project met objective. 
Parental Involvement 

• Parents of target students will establish regular contact with the school and the bilingual program 
through the services of the case manager and an educational assistant 

The case manager and educational assistant regularly phoned and wrote to project parents to discuss 
students' attendance and academic progress and to give information about the program. The family assistant also 
made regular home visits to project students. 

Project met objective* 

• Parents will be fully informed of the relevant aspects of ihe New York City school system, 
requirements for graduation, and the bilingual program through meetings and telephone calls. 

The bimonthly newsletter prepared by the resource teacher provided information to parents concerning 
the school and the New York City Public Schools system. During the spring semester parents attended monthly 
Parents' Association meetings, which included workshops on relevant topics. 

Project met objective* 

• Parents of the target students will participate in the Bilingual Advisory Committee. 

Seven parents participated on the Bilingual Advisory Committee, which met twice during the year* 
Project met objective. 




• A course in English as a Second Language will he offered to parents of program students. 
Another Title VII program offered E.S.L. classes in which Project DREAM parents participated. 
Project met objective. 



• A parent support group for parents of program students will be established by the case manager, 
group will bold monthly meetings on a specific topic. 

A parent support group was formed Monthly meetings were held following the Parents 1 Association 
meetings, and a variety of topics such as coping strategies, day care, and education were discussed. 

Project met objective. 

• A one-page bimonthly parent newsletter will be sent to parents of program students keeping them 
informed of all program and school activities. 

A bimonthly parent newsletter was prepared and sent to parents of students in the program. 

Project met objective. 

Other Activities 

The Technical Assistance Center (TAG) in the Bronx provided ten project students with computers to 
take home. The project selected recipients on the basis of their test grades, homework, attendance, and 
reliability. Hie students were required to write an autobiography, a family history, family biographies, and other 
assignments related to their background and culture. Parents were encouraged to use the computer at borne. 

Student achievements included: 

1. E.S.L. Spelling Bee, 1990, 2nd Place 

2. Physical Fitness Awards 

3. Science Fair, 1991, 1st Place 

4. Appearances on talent shows and bilingual program shows 

5. Participation in Spanish and E.S.L, Spelling Bees 

6. Articles in "El Vocero BflingOe" (Bilingual Program Newsletter). 



STUDENTS WITH SPECIAL ACADEMIC NEEDS 

Referral to special education was considered a last resort. The first step was usually to speak to the 
students* parents. Case conferences attended by the project director, case manager, and guidance counselors, 
were held biweekly to discuss these students. Students were observed and progress reports kept* If special 
education was deemed necessary, a referral would be made to the School-Based Support Team (S.B.S.T.) by the 
guidance counselor. The S.B.ST, included Spanish-speaking members. 



Number of Students Percentage 



Referred to special education: 
Referred to remedial programs: 
Referred to gifted and talented programs: 
Retained in grade: 



0 
0 
0 
57 



0 
0 
0 
12.6 




MA1NSTREAMING 

Seventeen students (3.8 percent of participants) were mainstreamed. 



CASE HISTORY 

I. was an eleventh-grade student in the program- She was 20 years old and had come to the United 
States from Mexico in September 1989. She had not attended school in Mexico for four years before emigrating 
because she had to care for her younger brothers and sisters. She had no knowledge of English when she entered 
Project DREAM but had progressed rapidly* She was an honor student and had the highest average in the tenth 
and eleventh grades, bilingual or mainstream* 
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APPENDIX A 



DATA COLLECTION AND ANALYSIS 

Student Data Form 

This one-page form is filled out by staff for each participating and mainstreamed student. OREA gathers 
data from this form on backgrounds, demographics, academic outcomes, attendance, referrals, and exit from the 
program. 

Project Director's Questionnaire 

The Project Director's Questionnaire includes questions on staff qualifications, program implementation, 
periods of instruction, and instructional materials and techniques. 

Project Director's Interview 

The interview gathers information on program and student or staff characteristics not supplied by the 
Project Director's Questionnaire. The interview also allows project staff to offer qualitative data or amplify 
responses to the Questionnaire. 

Citvwide Test Scores 

OREA retrieves scores centrally from the Language Assessment Battery (LAB) and other citywide tests. 
For evaluation purposes, these test scores are reported in Normal Curve Equivalents (N.CEs). N.C.E.S are 
normalized standard scores with a mean of 50 and a standard deviation (s.d.) of 21.06. They constitute an equal- 
interval scale in which the distance is the same between any two adjacent scores. A gain of 5 N.C.E.S is the same 
whether it is at the lower or the higher end of the scale. N.C.E.S can be used in arithmetic computations to 
indicate group progress. (Percentile scales, although more familiar to many, are unsuitable for such computations 
since they are not equal-interval.) 

1 .tkert-Tvpe Surveys 

Likert-type surveys, in which respondents mark their opinions on a scale from one to five, are used in a 
variety of ways. They examine student attitudes (i.e., toward school and career, native language use, and native 
and mainstream cultures). They also assess staff and parent attitude and reactions to workshops and other 
activities. 

Gap Reduction Evaluation Design 

OREA uses a gap reduction design for measuring changes in standardized tests. Since no appropriate 
non-project comparison group b available in New York City, where aU students of limited English proficiency 
(LEP) are entitled to receive supplementary services, OREA compares the progress of participating students with 
that of the group on which the test was nonned. It is assumed that the norm group would show a zero gain in 
the absence of instruction, and gains made by project students could be attributed to project services. 
(See "Citywide Test Scores" above.) 

To test whether pre/posttest gains are greater than could be expected by chance alone, OREA uses a 
t-test. To determine whether a difference between two proportions (e.g., program and mainstream attendance 
rates) is greater than could be expected by chance, OREA uses a z-test. The level of significance is set at .05 for 
all tests. 



APPENDIX B 



Instructanal Materials 

E.S.L 

Pathways to English. Books 1-4 - McGraw-Hill 

Pathways to Fflglisfr . Workbooks 1-4 - McGraw-Hill 

Gallery - Oxford University Press 

Line bv Line - Prentice Hall 

Longman Structural Readers - Longman, Inc. 

In The Beginning - Prentice Hail 

A Christmas Carol - Oxford Press 

The Adventures of Sherlock Holmes - Oxford Press 

Cloze Connections - Barnell Loft, Ltd. 

Easy Reading Selections in English • Prentice Hall 

Mobv Pick - Prentice Hall 

The Good Earth - Oxford Press 

Murders in the Rue Morgue - Prentice Hall 



N.LA. 

Asi Escribimos - National Textbook Company 

Asuntos Personates South-Western Publishing Company 

Bodas de Sangre Ediriones Catedras 

El Cuento Holt. Rinehart and Winston, Inc. 

Espafiol a Sentirlo McGraw-Hill, Inc. 

Fjggjgnes - Emece 

Fiesta al Noreste - Ediciones Destine 
Galena Hisoanica - McGraw-Hill, Inc. 

Literature Hispanoamericana - Holt, Rinehart and Winston, Inc. 

Mejora Tu Espafiol - Regents Publishing Company, Inc. 

Ortoerafia en AcckSn South- Western Publishing Company 

Pasajes - Random House, Inc. 

For Fronteras Culturales - Amsco School Publications 

Practicasde Oficina - South- Western Publishing Company 

San Mauel Bueno. Matn. v Tres Historias Mas - Coleccin Austral 

Segundo Libro - Amsco School Publications, Inc. 

Tesoro Hispanico - McGraw-Hill 

Workbook in Spanish fjfergg Years) - Amsco School Publications, Inc. 
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